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REMARKS 

Claims 1-80 were pending. Claims 1-80 were objected and rejected. Claims 1 and 79 are 
amended herein to more particularly point out and specifically claim subject matter which the 
applicant considers as the invention. Support for the amendment can be found in the 
specification as originally submitted, particularly on page 4, lines 31-34. Claims 81-89 are 
newly added. Dependent claims 81-84 are directed to two statutory classes, apparatus 
(system) and article of manufacture (computer readable medium), that implement the 
respective methods recited in independent claims 1 and 79. Support for the amendment can 
be found in the specification as originally submitted, particularly on page 56, line 33, through 
page 58, line 2, as well as drawings as originally submitted. Claims 85-86 are directed to a 
graphical user interface described in the specification and shown in Figure 23. Dependent 
claims 87-89 respectively recite limitations that properly narrow the scope of the base claim 
1 and intervening claims thereof. No new matter is introduced. By this Amendment, claims 
1-89 are pending. 

Regarding the Abstract of the Disclosure 

The abstract was objected to for exceeding 150 words. To obviate this objection, a new 
abstract within the range of 50 to 150 words is included in the substitute specification 
submitted herewith. The original abstract is hereby cancelled. 

Regarding the Specification 

The examiner has required a substitute specification with double spaced lines. The original 
specification has 12 point text with 18 point spacing and therefore complies to 37 CFR 1.52 
(b)(2)(i) which requires that the lines to be 1 1/2 or double spaced. To make reading easier 
for the examiner, however, a substitute specification reformatted with larger spacing is 
nonetheless submitted herewith. The substitute specification incorporates the aforementioned 
new abstract, since it would be redundant to submit a substitute abstract only to cancel and 
replace it with a new one. For similar reasons, the substitute specification incorporates the 
cross-reference to U.S. Patent No. 6,031,549, originally cited on page 57, line 10. No new 
matter is introduced or added. The substitute specification contains only subject matter from 
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the original specification and therefore is in compliance with 37 CFR 1.125(a) as set forth in 
the Office action. 

Regarding Claim Objections 

Claims 1-80 were objected to because they are not uniformly aligned. It is noted that line 
indentation is permitted under 37 CFR 1.75(i) and encouraged by MPEP 608.0 l(m), "[tjhere 
may be plural indentations to further segregate subcombinations or related steps." Further, 
since under the Revised Amendment Practice a listing of all pending claims is required when 
there is any amendment to a claim, it would be unnecessary to align the claims in the 
substitute specification. To comply with the examiner's requirement, however, the substitute 
specification submitted herewith is nonetheless reformatted so that all 80 claims are 
uniformly aligned. 

Regarding Claim Rejections 

Claims 1-7, 9-13, 15-25, and 27-79 were rejected under 35 U.S.C. § 102(e) as being clearly 
anticipated by Hatlelid et al. (U.S. Pat. No. 6,522,333 Bl, hereinafter referred to as 
"Hatlelid"). Claims 8, 14, 26, and 80 were rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Hatlelid. The rejections are respectfully traversed. The traversal to the 
rejections will be collectively discussed with respect to independent claims 1 and 79. 

"A claim is anticipated only if each and every element as set forth in the claim is 
found, either expressly or inherently described, in a single prior art reference." 
Verdegaal Bros. v. Union Oil Co. of California, 814 F.2d 628, 631, 2 USPQ2d 1051, 
1053 (Fed. Cir. 1987). "The identical invention must be shown in as complete detail 
as is contained in the ... claim." Richardson v. Suzuki Motor Co., 868 F.2d 1226, 
1236, 9 USPQ2d 1913, 1920 (Fed. Cir. 1989). 

Hatlelid's invention is hereby distinguished at least because Hatlelid clearly does NOT 
anticipate or suggest, inter alia, a computer-controlled agent . 
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One skilled in the computer art would have readily recognized that Hatlelid teaches a visual 
representation of a human end-user. This "visual representation" DOES NOT and is 
INCAPABLE of having its own personality or behavior, lack of intelligence notwithstanding. 

Contrastingly, the computer-controlled agent according to the present invention not only has 
intelligence, it is capable of CARRYING ON CONVERSATIONS WITH HUMANS 
USING NATURAL LANGUAGE [See, e.g., original Spec, page 7, line 34, and page 9, lines 
2-10 and 25-34]. 

Computer-controlled agents might have "visual representations." However, such "visual 
representations" represent the agents themselves and NOT human end-users. Therefore, the 
examiner's statement that "Hatlelid discloses a means for authoring content of a computer- 
controlled agent, wherein this agent is used to visually represent a user" is incorrect. The 
computer-controlled agent INTERACTS WITH A USER and is not used to visually 
represent a user . Moreover, one skilled in the computer art would certainly appreciate that 
writing application modules for visual representation cannot anticipate or suggest authoring 
content of a computer-controlled agent - much like a manual of how to master a puppet 
would not have anticipated or suggested how to teach Pinocchio to talk and behave. To 
interpret otherwise requires a giant leap that is far from reasonable. 

The fundamental, substantial differences between a visual representation and a computer- 
controlled agent not only distinguishes Hatlelid from the claimed invention, it also renders 
the rest of the rejections moot, if not inapplicable. For example, in Hatlelid, a user selects a 
personality type and behavioral characteristics to be associated with the user's visual 
representation [Col. 6, lines 15-16 and 64-67, cited by the examiner]. The user does NOT 
author contents of the personality type or behavioral characteristics. That user is a user and 
NOT AN AUTHOR . Further, the "personality type" taught by Hatlelid does not fall within 
the meaning of "a potential context" specifically defined in the present application [Original 
Spec, page 5, lines 6-12]. 
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Until the present application, to complete a particular computer-controlled agent usually 
requires a substantial amount of the detailed logic that is characteristic of computer 
programming [Original Spec, page 2, lines 17-19]. Hatlelid neither solves nor addresses this 
issue. In contrast, the present application advantageously enables even a non-technical person 
to directly and independently develop content for interactive agents without the need to 
acquire technical skills or depend upon significant support from technical experts [id. at lines 
19-33]. 

The following analysis focuses on three foundational differences, which individually and in 
combination, categorically distinguish all pending claims from Hatlelid' s teaching. As 
discussed herein, these claims are not anticipated by Hatlelid, are not obvious over Hatlelid, 
nor are they supported by Hatlellid's disclosure. 

The key differences lie in (a) the different functions of the two inventions; (b) the different 
uses of the term "context" in the two inventions; and (c) the different scope of the behavior 
that is automated by the two inventions. 

Hatlelid 's invention teaches (a) "a method of communicating data from a user to a remote 
recipient," (b) the term "context" refers to additional information the user can provide "to the 
recipient for interpreting the communicated data," and (c) the automated behavior consists of 
instantiating a "behavioral movement of the visual representation" of the user, based on an 
immediate communication from the user. 

By contrast, the present invention teaches (a) "a method for authoring content of a computer- 
controlled agent," (b) the term "context" refers to either a potential combination of state 
variables which might occur during an operation of the agent (a "potential context") or a 
combination of actual state variables which does occur during a particular operation of the 
agent (an "actual context"), such that the author can provide content to be used in controlling 
a behavior of the agent in the event that a particular actual context matches a particular 
potential context, and (c) the automated behavior may include, for example: detecting actual 
contexts; determining whether a given actual context matches a particular potential context; 
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instantiating or constructing a behavioral movement of the visual representation of the agent; 
selecting, instantiating, or constructing a line of dialog for the agent; operating a browser of 
other application in the recipient's environment; getting or putting information n a database; 
setting or evaluating a state variable. 

Below further explains and illustrates these differences between the two inventions. 

Hatlelid's invention teaches a method of communicating data from a user to a remote 
recipient, by enabling the user to select and send a "behavioral characteristic" that causes a 
visual representation of the user, the "user-avatar," to perform associated behavioral 
movements in the recipient's environment. The user may formulate and send other data (e.g., 
a verbal message) to be delivered to the recipient concurrently with the user- avatar's 
performance of the behavioral movements. The user-avatar's behavioral movements 
presumably provide an interpretation-context that enriches the recipient's interpretation of 
the user's communication. The user may engage in an extended communication sequence 
with the recipient, taking into account the recipient's intervening responses as he or she 
wishes, while formulating the data and selecting the behavioral characteristic for each of his 
or her own successive communications. 

Here is a simple example of Hatlelid's invention: 

• A communication system provides a set of behavioral commands (e.g., wave, nod) and 
behavioral characteristics (e.g., enthusiastic, friendly), which a user can combine to tell its 
user-avatar to perform each command according to each characteristic; 

• During a communication session, a user Dave recognizes a newly connected recipient as his 
friend John. Intending to send an enthusiastic greeting, Dave formulates and sends John a 
communication comprising: the data "Hey John!" the behavioral command wave, and the 
behavioral characteristic enthusiastic^ 

• The communication is delivered to John, with the data "Hey John!" printed in a text area 
concurrently with movement of Dave's user-avatar to perform an enthusiastic wave. 

• Observing Dave's user-avatar's enthusiastic wave as a context for his "Hey buddy!" data, 
John interprets Dave's communication as an enthusiastic greeting. 

• John responds to Dave, "Hey Dave!" 
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• John and Dave continue their communication as they wish. 

By contrast, the present invention teaches a method for authoring content of a computer- 
controlled agent by enabling an author to describe a prospective-context and create 
associated content for use in controlling any aspect of the agent's behavior in an actual- 
context that matches the prospective-context and occurs during an operation of the agent. 
Note that the present computer-controlled agent takes the place of Hatlelid's human user in a 
communication with a recipient. Like the human user, the computer-controlled agent can 
select (or construct) and send a behavioral description that causes a visual representation of 
the agent, the "agent-avatar," to perform associated behavioral movements (or other 
behaviors) in the recipient's environment. Also like the human user, the agent may formulate 
and send other data (e.g., a verbal message) to be delivered to the recipient concurrently with 
the agent-avatar's performance of the behavioral movements. Similarly, the agent-avatar's 
behavioral movements can provide an interpretation-context that enriches the recipient's 
interpretation of the agent's communication. Again like the human user, the agent may 
engage in an extended communication sequence with the recipient, taking into account the 
recipient's intervening responses as it wishes, while formulating the data and behavioral 
description for each of its own successive communications. 

Because the present agent is computer-controlled, it can communicate with a recipient 
without run-time participation by its human author. In fact, it can communicate concurrently, 
asynchronously, and independently, with multiple recipients, at arbitrary times and 
circumstances, as determined by events outside the direct control or knowledge of the author. 

Here is a simple example of the present invention: 

• An authoring system provides a means of specifying behavior commands, dialog 
commands, behavior/dialogue characteristics, and database operations, and using 
those specifications, in some cases with parameters that can take on different values, 
to specify prospective contexts and associated content for an agent; 

• Intending to create an agent named "Harmony" that can personalize her greeting 
behavior for individual recipients, an author: 
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(a) specifies a set of behavior commands (e.g., wave, nod), parameterized dialog 
commands (e.g., greet #recipient t #introduce~self-to ^recipient), where #recipient 
performs a database look-up for the specified recipient's name or, if it does not 
find the recipient, defaults to "friend," and behavior/dialog characteristics (e.g., 
enthusiastic, friendly), such that the agent-avatar can perform each behavioral or 
dialog command according to each characteristic; 

(b) further specifies that, when a recipient initiates a communication link to 
Harmony, she should check her database to see if she recognizes (has a record 
for) the newly linked recipient and then: 

i. if she recognizes the recipient, send a communication to the recipient 
comprising: the dialog command greet ^recipient, the behavior command 
wave, and the behavior/dialog characteristic enthusiastic, 

ii. if she does not recognize the recipient, send a communication to the recipient 
comprising: the dialog command greet ^recipient, the behavior command nod, 
and the behavior/dialog characteristic friendly. 

Later, while the author is sleeping, two recipients, Jane and Mary, initiate 
communication links to Harmony in close succession. She communicates with them 
concurrently, asynchronously, and independently, as follows: 



Recognizing Jane - 

Harmony sends a communication comprising: the dialog command greet ^recipient, the 

behavior command wave, and the characteristic enthusiastic. 
The communication is delivered to Jane, with the data "Hi Jane!" printed in a text area, while 

the Harmony-avatar performs an enthusiastic wave. 
Observing the Harmony- avatar's wave as a context for her "Hi Jane!" data, Jane interprets 

Harmony's communication as an enthusiastic greeting. 
Jane responds, "Hi Harmony, how are you?" 
Jane and Harmony continue as they wish. 
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Not recognizing Mary - 

Harmony sends a communication comprising: two dialog Commands greet ^recipient and 
ftintroduce-selfto ^recipient, the behavior command nod, and the characteristic 
friendly. 

The communication is delivered to Mary, with the data "Hello friend. Please let me introduce 
myself. I'm Harmony, a computer-controlled agent. If you don't mind, friend, may I 
ask you to introduce yourself?" printed in a text area, while the Harmony-avatar 
performs a friendly nod. 

Observing the Harmony-avatar's friendly nod as a context for her "Hello friend..." data, 
Mary interprets Harmony's communication as a friendly greeting. 

Mary responds, "Hi Harmony. I'm Mary, a human being." 
Mary and Harmony continue as they wish. 

In summary, as these examples illustrate, Hatlelid's invention is categorically different from 
the claimed invention. Hatlelid's invention allows a human user to directly coordinate the 
movement of a visual puppet to complement the user's own live dialog with a recipient. By 
contrast, the present invention allows a human author to create a computer-controlled agent 
that can then operate autonomously and without participation by the author and taking the 
place of the human user, generating the dialog, coordinating the movement, and controlling a 
variety of other behaviors for a number of its own puppets, each of which communicates 
independently with a different recipient. 

Regarding Prior Art of Record Not Relied Upon 

The Office action did not articulate or explain how Matsuda et al. (U.S. Pat. No. 6,466,215, 
hereinafter referred to as "Matsuda") or Cheng (U.S. Pat. No. 6,329,986) apply to the 
claimed invention. Under 37 CFR 1.1 1 1(b), "[t]he reply must present arguments pointing out 
the specific distinctions believed to render the claims, including any newly presented claims, 
patentable over any applied references ." (emphasis added). Although the Office action did 
not apply either of these two references to any of the pending claims, the Applicant 
nevertheless submits the following. 
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Regarding Matsuda 

Matsuda's invention enables automated creation of a story animation out of parts. Given the 
labor-intensive process of creating story animations "by hand," Matsuda's invention allows 
animation parts to be stored with attribute-value information and then selected and sequenced 
based on attribute-values specified in a story. It also allows for an animation part selected for 
one story step to be accepted or rejected, based on arithmetic evaluation of its connectivity 
from the animation part selected for the preceding step. 

By contrast, the present invention allows an author to create a computer-controlled agent that 
can operate in real time, generating its own dialog, movements, and other behaviors and 
responding to the dialog, movements, and other behaviors of a user or other interactive 
entity. 

Thus, the two inventions are different. At best, they are complementary. In principle, they 
could be used in combination. For example, using the present invention, an author could 
create a computer-controlled agent that, among other things, generates a "story" it wishes to 
perform, encodes the story in a sequence of attribute-values, and then calls upon a 
mechanism such as Matsuda's to generate an appropriate sequence of animation parts, which 
it would then execute. In practice, however, this combination would not be feasible unless 
Matsuda's mechanism could operate in real time. 

Regarding Cheng 

Cheng's invention enables priority-based communication and rendering of objects, including 
user-avatars, in a virtual environment. Given the typical competition for communication and 
rendering resources, Cheng's invention dynamically prioritizes objects for a given user, 
based on parameters such as relative positions of objects, user profile information, social 
relations among users, etc. At each point in time, user-specific object priorities determine 
which objects are communicated or rendered and the quality of that communication or 
rendering. 
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By contrast, the present invention allows an author to create a computer-controlled agent that 
can generate its own dialog, movements, and other behaviors and respond to the dialog, 
movements, and other behaviors of user-avatars, other agents, or other objects that may be 
present. 

Thus, the two inventions are different. At best, they are complementary. For example, using 
the present invention, an author could create an agent that operates in a virtual environment. 
In that case, it might compete with other objects for communication and rendering resources 
and some mechanism, such as Cheng's, might be used to allocate resources among 
competing objects. 



Conclusion 

For the foregoing reasons, it is respectfully submitted that the present invention is patentably 
distinct from, not anticipated by, and unobvious in view of Hatlelid. It is further respectfully 
submitted that independent claims 1, 79, and 85 respectively recites subject matter not 
reached by the closest prior art of record under 35 USC §§ 102(e) and/or 103(a). 
Accordingly, independent claims 1 and 79 are submitted to be patentable. 

Reliance is placed on In re Fine, 5 USPQ 2d 1596, 1600 (Fed. Cir. 1988) and Ex parte 
Kochan, 131 USPQ 204 (Bd. App. 1960) for allowance of the dependent claims 2-78, 80-84, 
and 86-89, since they differ in scope from their respective parent independent claims 1, 79, 
and 85 which are submitted to be patentable. 

This Response/ Amendment is submitted to be complete and proper in that it places the 
present application in a condition for allowance without adding new matters. Since the 
examiner has done a thorough search in the first Office action in light of the entire 
application disclosure and claims, no new search would be necessary. Favorable 
consideration and a Notice of Allowance of all pending claims are therefore earnestly 
solicited. The examiner is sincerely invited to telephone the undersigned at 650-331-8413 for 
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discussing an examiner's Amendment or any suggested actions for accelerating prosecution 
and moving the present application to allowance. 



Respectfully submitted, 




Katharina Wang Schuster, Reg. No. 50,000 
Attorney for the Applicants under 37 CFR 1.34 



Lumen Intellectual Property Services 
2345 Yale Street, Second Floor 
Palo Alto, CA 94306 

(O) 650-424-0100 (F) 650-424-0141 x 8413 
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